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Fourier  Optical  Signal  Processors 


1.  INTRODUCTION 


Fourier  optical  signal  processing  is  certainly  the  oldest  form  of  optical  signal  processing  with  roots  going 
back  to  the  last  century  with  the  works  of  Abbe  in  1893.  By  Fourier  optical  signal  processing,  we  mean  the 
intentional  modification  of  the  spectrum  of  an  image  to  modify  that  image  or  some  process  based  on  that  image 
(such  as  image  restoration)  or  recognition  of  that  image  by  an  autonomous  system.  It  is  this  latter  application 
that  we  will  deal  with  at  some  length  in  this  article.  There  are  two  basic  reasons  for  that.  Pattern  recognition 
has  not  been  “solved”  by  any  stretch  of  the  imagination,  either  by  digital  or  optical  processing.  However,  with 
new  algorithms  and  devices  we  think  a  hybrid,  optical/digital  system  can  make  a  significant  advance  in  this  area, 
in  the  form  of  a  system  that  is  ready  for  transitioning  to  the  work  place. 

The  basic  premise  of  the  Fourier  optical  processor  is  the  ability  of  a  lens  to  extract  a  two-dimensional 
Fourier  transform  of  an  input  signal,  as  shown  in  Figure  1.  Note  that  the  input  illumination  must  be  highly 
coherent  light,  as  from  a  laser,  and  that  the  computation  is  done  fully  in  parallel,  literally  at  the  speed  of  light, 
about  1  ft/ns.  This  is  equivalent  to  a  throughput  of  about  4  •  lO1^  mathematical  opcrations/scc  for  a  256  x  256 
sample  data  array  in  a  Fourier  optical  system  approximately  l  ft  long,  using  a  20  mW  lasei  diode  light  source. 
This  estimate  is  based  on  the  number  of  digital  multiplications  and  additions  required  to  perform  the  same 
transform  using  an  FFT  algorithm.  It  is  this  vast  “number  crunching”  potential  of  optical  processing  that  has 
stimulated  and  sustained  research  and  development  in  this  area  since  the  advent  of  the  laser  as  a  practical  source 
of  such  a  system.  Up  until  the  past  few  years,  the  bottleneck  in  using  this  concept  for  anything  practical  has  been 
getting  the  data  into  and  out  of  the  system.  Fortunately,  recent  developments  in  hardware  and  software  (filter 
algorithms),  have  mitigated  this  problem,  and  spurred  a  Hurry  of  research  and  development  activity  in  this  field. 


(Received  for  publication  17  April  1990) 
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2.  BACKGROUND 


In  1964,  Vandcr  Lugt  proposed  using  a  Fourier  plane  mask  for  pattern  recognition, *  and  this  architecture 
is  usually  referred  to  as  a  frequency  plane  correlator.  His  system  performed  a  correlation  between  two  functions, 
fl  and  12,  and  is  based  on  the  autocorrelation  theorem  and  the  Fourier  transform  property  of  a  lens  and  coherent 
light: 

C(x)  =FT-*  {FjM-FSM),  (1) 

where  C(x)  is  the  correlation  function,  FT  stands  for  the  Fourier  transformation,  Fl  and  Fl  arc  the  Fourier 
transforms  of fl  and  fl  respectively,  and  the  *  denotes  complex  conjugation.  We  will  use  the  convention  of  let¬ 
ting  “x"  represent  x,  y,  “v”  for  v #  Vy,  etc.  This  is  implemented  by  the  Fourier  transform  system  of  Figure  1.  The 
function  F|  is  equivalent  to  a  matched  filter  and  is  a  complex  function  expressible  in  terms  of  magnitude  and 
phase: 


f2=  |  f2  |  -S\ 


(2) 


Vandcr  Lugt’s  insight  was  realizing  a  way  to  write  this  complex  function  on  an  energy-sensitive  medium, 
photographic  film.  The  method  proposed  was  holography,  which  results  in  encoding  the  complex  function  on  a 
spatial  carrier,  and  involves  an  object  beam  modulated  by  the  reference  pattern.  With  the  introduction  of 
computer-generated  holography,^- 3  (CGH),  it  became  possible  to  make  matched  filters  of  mathematically 
synthesized  objects,  but  the  CGH  filter  still  was  written  on  photographic  film,  and  therefore  could  not  be  written 
in  real  time. 

Another  optical  architecture,  which  has  become  known  as  the  joint  transform  correlator  (JTC),  was  pro¬ 
posed  several  years  later  by  Weaver  and  Goodman,'*  and  is  shown  in  Figure  2.  Here  the  reference  function  and 
unknown  object  arc  presented  together  in  the  input  plane,  and  their  combined  or  joint  Fourier  transform  is 
produced  in  the  focal  plane  behind  the  first  lens.  If  the  joint  Fourier  transform  is  recorded  on  a  square-law  or 
energy  detector,  originally  photographic  film,  and  the  inverse  Fourier  transform  is  taken,  we  sec  that  again  we 
have  implemented  Eq.  (1)  and  have  produced  a  matched  filter  correlation  between  the  two  signals.  One  advan¬ 
tage  of  this  system  is  that  one  does  not  need  a  matched  filter;  one  simply  introduces  the  reference  function, 
usually  a  real  function,  beside  the  input  pattern  at  the  input  plane.  This  contrasts  with  frequency  plane  correla¬ 
tion  which  requires  producing  a  Fourier  transform  of  the  reference  function,  and  encoding  and  recording  this 
complex  function  as  a  physical  matched  filter  to  implement  modulation  in  the  Fourier  plane.  A  disadvantage  of 
the  joint  transform  correlator  is  that  the  space-bandwidth  product  available  in  the  input  plane  must  be  shared  by 
the  input  and  reference  signals,  as  well  as  a  guard  band  to  ensure  separation  of  correlation  patterns  from  unde- 
sired  terms  at  the  output  plane.  This  architecture  recently  has  attracted  much  interest  as  researchers  have  re¬ 
placed  the  photographic  film  in  the  Fourier  plane  with  CCD  detectors,  and  the  input  film  with  a  miniature  liquid 
crystal  TV  (LCTV)  display,3-  6  thus  yielding  real-time  operation.  In  addition,  an  architecture  has  been  proposed 
which,  by  using  a  time-sequencing  scheme,  uses  only  a  single  spatial  light  modulator  (SLM)  for  both  the  input 
and  Fourier  plancs7-  ®  This  greatly  reduces  the  size  and  cost  of  this  correlation  system. 

These  then  arc  the  basic  Fourier-optical  pattern  recognition  architectures  we  have  to  work  with  —  based 
on  the  simple  principle  that  a  lens  focusing  a  beam  of  coherent  (laser)  light  modulated  with  an  image  produces  a 
2D  optical  analog  Fourier  transform  of  that  image.  Recent  developments  in  hardware  (SLMs),  and  new  filter 
algorithms  particularly  amenable  to  optical  iuijm.itioHtulion  (phase-only  fillers),  have  combined  to  create  new 
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Figure  2.  Joint  transform  correlator.  In  (a),  the  joint  Fourier  spectrum  is  recorded.  In  (b),  the  joint  spectrum  is  re- 
illuminated  and  Fourier  translormed  to  produce  the  cross  correlation  between  a  reference  (R)  and  signal  (S). 


interest  and  potential  in  this  technology. 


3.  REAL-TIME  IMPLEMENTATION  ISSUES  AND  NEW  FILTER  ALGORITHMS 


Real-time  implementations  arc  critical  to  the  practical  application  of  optical  correlation,  and  the  key  device 
technology  is  the  spatial  light  modulator  (SLM),  which  introduces  signal  or  image  patterns  as  either  correspond¬ 
ing  light  amplitude  or  phase  modulation  patterns  across  the  optical  beam.  Spatial  light  modulator  technology 
will  not  be  reviewed  here,  but  it  is  fair  to  say  that  the  performance  of  these  devices  limits  the  performance  of 
correlators,  particularly  with  regard  to  space-bandwidth  product  and  speed.  Two  important  categorizations  of 
SLMs  are  addressing  method  (electrical  or  optical),  and  modulation  type  (phase  and/or  amplitude;  Correlator 
systems  need  SLMs  at  both  input  and  Fourier  planes,  except  as  noted  a  hove, ^  ^  and  the  requirements  differ 
depending  on  the  application. 

About  a  decade  ago,  a  device  to  replace  photographic  film  as  the  input  means  in  the  correlator  of  Figure  1. 
named  the  Liquid  Crystal  Light  Valve,  was  made  commercially  available  by  Hughes  Aircraft  Corp.  It  was  an 
optically  addressed  incoherent-to  coherent  image  converter,  and  as  such  could  take  an  incoherent  image,  as 
from  a  CRT  or  imaging  lens,  and  use  it  to  spatially  modulate  a  beam  of  laser  light.  Now  for  the  first  time  real¬ 
time  inputs  could  be  used  with  the  optical  correlator. 

Probably  the  most  challenging  SLM  requirement  is  that  of  real-time  matched  filter  modulation  at  the 
Fourier  plane  of  a  Vandcr  Lugt  correlator.  The  matched  filter  generally  is  a  complex  function,  requiring  inde¬ 
pendently  controlled  phase  and  magnitude  spatial  modulation.  Most  SLMs,  such  as  the  LCLV  in  its  normal 
mode  of  operation,  provide  only  magnitude  modulation,  sometimes  accompanied  by  incidental  phase  modula¬ 
tion  which  is  not  independently  controllable.  Both  Vandcr  Lugt  and  Goodman  realized  clever  ways  to  record 
the  complex  filter  on  an  intensity  or  amplitude-only  medium,  for  example,  photographic  film  However,  film  is 
not  a  recording  medium  that  allows  for  quick,  easy,  or  simple  writing  of  new  or  modified  filters,  as  is  necessary 
for  searching  through  an  array  of  reference  functions.  The  result  was  that  optical  correlation  did  not  find  much 
application  in  the  real  world. 

The  resolution  and  space-bandwidth  product  of  the  LCLV  and  other  candidate  SLMs  are  not  sufficient  to 
support  practical  holographic  matched  filters.  However,  with  Lohmann's  simplified  version  of  the  true  holo¬ 
graphic  (Vandcr  Lugi)  filter,  [the  computer-gonermed  holographic  (CGH)  filter],  it  is  row  possible  «--;»e  very 
low  resolution  reference  images  on  current  SLMs.  The  state-of-the-art  space-bandwidth  product  in  pixellatcd 
devices  is  roughly  256  x  256  individually  addressable  elements.  One  problem  with  this  is  the  very  low  diffraction 
efficiency  of  the  SLM  when  used  as  a  holographic  filter.  Most  of  the  light  goes  into  the  zeroth  order,  a  residual 
on-axis  DC  spike.  We  will  now  discuss  some  recent  developments  that  greatly  facilitate  implementation  of  fil¬ 
ters. 

3.1  Phase-Only  Filters 

As  wc  have  seen,  practical  application  of  the  matched  filter  in  the  past  was  limited  because  the  filter  is  a 
complex  quantity,  and  no  effective  means  exists  to  implement  such  a  filter  in  real  lime.  Previous  work  in  image 
analysis  and  synthesis  had  shown  that  the  phase  information,  in  the  Fourier  plane,  was  significantly  more  impor¬ 
tant  than  the  amplitude  information  in  image  reconstruction  9’  '*'  Caulfield'*'  and  Horner'  ’  originally  con¬ 
ceived  the  idea  of  using  only  the  Fourier  phase  of  the  signal  to  make  a  matched-like  filter,  as  reported  by  Caul- 
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field.  Horner  and  Gianino*^  first  showed  the  feasibility  of  this  idea  through  eomputer  simulation,  naming  it  the 
“phase-only  filter”  (POF).  A  much  earlier  paper*-*  treated  the  case  in  which  the  amplitude  information  was 
omitted  by  hard-clipping  from  a  holographic  matched  filter.  The  authors  concluded  that  in  a  filter  with  “only 
phase  control”  the  SNR  in  the  correlation  plane  actually  decreased  by  29  percent.  To  our  knowledge,  this  is  the 
only  mention  in  the  literature  of  a  correlation  filter  using  only  phase  until  References  lOand  12.  The  computer 
simulations  showed  that  a  phase -only  filter  would  indeed  compare  quite  favorably  to  the  matched  filter  in  that 
the  structure  of  the  correlation  peak  was  typically  a  single  narrow  spike  with  negligible  side  lobes.  *2  The  com¬ 
puter  simulations  also  showed  that  the  correlation  peak  was  anywhere  from  50  to  500  times  higher  than  that  of 
the  matched  filter.  A  simulation  for  a  matched  filter  is  shown  in  Figure  3(a),  and  the  phase-only  filter  in  Figure 
3(b).  The  input  signal  was  the  letter  ‘G’.  For  an  optical  system  to  have  a  pure  phase  element  in  the  filter  plane 
made  good  sense  because  all  the  light  gets  through  without  attenuation.  Everyday  examples  of  this  are  lenses 
and  prisms.  In  contrast,  the  amplitude  portion  of  a  matched  filler  attenuates  the  light  since  it  is  written  on  a 
passive  device  such  as  film  or  an  SLM,  and  the  peak  transmission  cannot  exceed  11X)  percent,  which  usually 
occurs  at  the  origin  of  the  frequency  plane.  Also  the  light  budget  of  a  holographically-recorded  matched  filter  is 
further  eroded  by  diffraction  efficiency,  particularly  if  one  tries  to  implement  it  on  a  spatial  light  modulator. 
Thus,  the  100  percent  light  efficiency  of  the  POF  is  a  distinct  advantage  in  a  low-powered  optical  correlator 
where  system  and  detector  noise  could  be  a  problem.  There  are  several  sources  of  noise  in  an  optical  correlator. 
One  is  the  so-called  artifact  noise.  Since  we  use  coherent  light,  any  slight  imperfections  in  the  lenses,  beam  split¬ 
ters,  SLM,  etc.  generate  their  own  diffraction  patterns  and  propagate  through  to  the  correlation  plane.  Another 
source  of  noise  in  systems  with  insufficient  laser  source  power  is  detector  noise.  Figure  3  also  shows  the  large 
increase  in  correlation  peak  intensity  obtained  with  the  phase-only  filter  —  a  factor  of  over  300  when  compared 
to  the  matched  filter  for  the  particular  case  studied.  This  corresponds  to  the  case  where  the  input  laser  energy 
and  pattern  arc  kept  constant  and  only  the  filter  type  is  changed.  The  light  through  the  phase-only  filter  has  a 
much  better  chance  of  breaking  through  the  residual  noise  background  and  being  detected.  The  reason  for  the 
considerably  higher  peak  is  simple  conservation  of  energy.  First,  with  the  POF,  all  the  energy  gets  through  the 
filter  plane  (Figure  1)  since  there’s  no  amplitude  function  to  attenuate  it;  second,  the  energy  that  does  get 
through  to  the  correlation  is  concentrated  in  a  much  narrower  peak,  approximating  a  delta  function  in  most 
cases. 

Another  advantage  of  the  POF  is  the  reduction  in  data  required  to  store  a  filter  because  only  the  phase  is 
retained.  The  Vandcr  Lugt  matched  filter  uses  a  carrier  frequency  to  encode  the  complex  amplitude  informa¬ 
tion.  A  phase-only  filter  does  not  require  a  carrier  or  encoding  scheme  to  write  or  record  it,  as  it  is  a  pure  phase 
entity.  This  results  in  a  large  savings  in  space-bandwidth  product  (SBP).  For  example,  an  eight-level  Lohmann 
type  computer-generated  hologram  pattern*  encoding  a  128  x  128-sample  matched  filter  requires  131  kb  (kilo¬ 
bytes  of  storage.  The  same  filter  as  a  POF  requires  only  16  kb.  In  the  next  section  we  w  ill  sec  that  we  can  further 
reduce  this  by  going  to  the  binary  POF,  which  will  require  only  2  kb. 


3.2  Binary  Phase-Only  Correlation 

Shortly  after  the  phase-only  filter  was  proposed,  a  simplification  of  this  concept  was  proposed  independent 
!y  by  two  researchers.  Psaltis  et  al*1*  showed  experimental  results  using  a  symmetric  input  test  signal  and  writing 
its  corresponding  bipolar  Fourier  transform  on  a  binary  SLM.  Through  a  simple  but  clever  modification  they 
adapted  the  SLM. "  hich  was  designed  to  write  binary  amplitude  modulation,  to  write  binary  phase  functions, 
that  is,  c,7T  and  e'(*  or  -1  and  +1.  Horner  and  Legcr**  showed  through  computer  simulation  that  arbitrary  sig- 


nals  having  no  particular  symmetry  could  be  Fourier  transformed,  “brute  force”  binarized  and  the  resulting 
binary  phase-only  filter  (BPOF)  used  for  pattern  recognition.  The  basic  idea  is  shown  in  Figure  4,  where  the 
continuous  phase  of  the  Fourier  transform  is  shown  in  part  (a).  Curve  (b)  shows  the  binary  signal  derived  from 
Figure  4  (a)  by  setting 


=  0  degrees,  Fr(v)>  0 

4>BPOF{  ,  (3) 

=  1 80  degrees,  Fr(v)<Q 

where  Fr(v)  stands  for  the  real  part  of  the  Fourier  transform. 

It  should  be  noted  that  the  correlation  plane  output  of  an  input  correlated  with  a  BPOF  consists  of  both  a 
correlation  and  a  convolution  signal.  To  see  how  this  comes  about,  consider  a  sample  point  in  the  Fourier  plane 
with  a  phase  angle  </>,•  as  shown  in  Figure  5.  We  can  make  this  point  binary  by  adding  a  phasor  at  a  conjugate 
angle  as  shown  by  the  dashed  line.  The  sum  of  these  two  phasors,  normalized  to  unity,  is  a  phasor  at  zero  angle. 

A  similar  argument  applies  to  the  left  half-plane.  Doing  this  at  every  point  in  the  Fourier  plane  produces  a  func¬ 
tion  equivalent  to  a  BPOF, 

H(v)  =  k  ■  +  k  ■  e'ty  ,  (4) 

where  k  =  1/(2  •  cos(<f>)).  The  impulse  response  of  this  function  is  the  Fourier  trarsform  of  Eq.  (4).  The  Fourier 
transform  of  ^  gives  an  edge -enhanced  version  of  the  original  signal,  and  the  transform  of  c'^  gives  an  invert¬ 
ed  edge-enhanced  likeness  of  the  original  signal.  Viewing  correlation  in  terms  of  a  shift,  multiply,  and  integrate 
process,  we  can  see  that  the  inverted  (180-degree  rotated)  component  is  responsible  for  the  convolution  while 
the  normal  image  yields  correlation.  A  photograph  of  the  impulse  response  of  a  BPOF  is  shown  in  Figure  6(b). 
The  filter  was  fabricated  on  a  fused  quartz  substrate  using  VLSI  and  reactive-ion  etching  techniques.16  The 
image  from  which  it  was  made  is  shown  in  Figure  6(a).  An  interesting  technique  has  been  proposed  for  reducing 
the  convolutional  response  by  incorporating  a  random  phase  mask  in  the  filter  function  and  at  the  Fourier 
plane.17 

The  choice  of  angle  of  the  Fourier-plane  binarizalion  axis  used  to  generate  a  BPOF  was  studied  by  Kail- 
man1^  who  showed  that  if  the  axis  was  allowed  to  rotate  and  a  BPOF  made  for  each  case,  the  output  SNR  would 
go  through  a  maximum.  This  concept  was  further  investigated  and  the  name  "threshold  line  angle”  (TLA)  was 
proposed  for  the  binarization  axis.19-  20  The  filter  type  investigated  by  Psaltis14  involved  thresholding  on  the 
real  part  of  the  transform,  corresponding  to  threshold  line  angle  =  0,  and  is  called  a  cosinc-BPOF  because  it  can 
be  derived  from  only  the  cosine  transform.  This  BPOF  type  embodies  information  concerning  only  the  even  pari 
of  the  reference  pattern.  The  sinc-BPOF  is  defined  with  threshold  line  angle  =  90  degrees,  corresponding  to 
thresholding  on  the  imaginary  part  of  the  Fourier  transform,  and  was  first  used  by  Horner  and  Lcger.1-*’  It 
encodes  only  odd-part  information  from  the  reference  pattern.  Hartlcy-BPOFs19  result  from  thresholding  on 
the  Hartley  transform  or  equivalently  by  thresholding  on  the  Fourier  transform  with  threshold  line  angle  =  45 
degrees. 20  This  type  BPOF  encodes  a  mixture  of  equal  components  of  even  and  odd  reference  pattern  informa¬ 
tion.  Flannery  et  al20  studied  BPOF  performance  using  a  tank  embedded  in  a  cluttered  background  and  found  a 
variation  of  about  2  db.  ir  signal-to-clutter  ratio  when  the  threshold  line  angle  was  varied  over  its  full  range. 
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Figure  5.  Binarization  of  continuous  phase  function  by  the  addition  of  its  complex  conjugate. 
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3.3  Noise  Performance  of  Phase-Only  Filters 

The  theoretical  noise  performance  of  POFs  and  BPOFs  relative  to  the  classical  matched  filter  has  been 
investigated.  The  results  depend  on  the  definition  of  signal-to-noise  ratio  (SNR),  and  the  type  of  noise  used. 

The  POF  with  an  appropriate  choice  of  bandpass  has  been  shown  to  be  the  optimum  phase-only  filter  function 
for  classical  noise  (white,  additive,  zero  mean,  Gaussian)  with  typical  SNR  figures  of  5  db  less  than  the  classical 
matched  filter.2*’  22  However,  when  the  sidelobes  of  the  correlation  pattern  are  included  in  the  noise  metric,  as 
originally  proposed  by  Horner  et  al,24  the  POF  and  BPOF  inevitably  have  higher  SNRs  than  the  matched  filter. 
Recently  Dickey  and  Romero2^  showed  that  the  POF,  in  the  original  form  proposed,*2  is  the  optimum  of  all 
possible  unity-amplitude  filters  (that  is,  generalized  POFs),  with  regard  to  a  metric  they  called  peak-to-sidelobe 
ratio;  it  is  nearly  identical  to  the  SNR  defined  by  Horner  et  al,24  for  correlation. 

Dickey  and  Romero2^  also  established  a  dual  optimality  of  the  POF  or  unity-amplitude  filter.  The  latter 
term  implies  a  more  general  definition  of  the  POF  whereby  any  phase  value  can  be  assigned  to  a  filter  element 
by  an  arbitrary  formulation.  In  discussion  up  to  this  point,  the  phase -only  filter  was  defined  in  the  sense  that  it 
was  the  phase  derived  specifically  from  the  Fourier  transform.  The  more  general  meaning  of  phase-only,  that  is, 
unity-amplitude,  will  also  be  used  later  in  this  report,  and  the  choice  of  specific  or  general  meaning  of  phase -only 
should  be  clear  from  context. 

Dickey  et  al26  have  established  a  lower  bound  of  -6  dB  for  the  signal-to-noise  performance  of  the  Hartley- 
BPOF  relative  to  an  optimal  POF,  and  an  upper  bound  of  3  dB  for  the  improvement  over  the  Hartley  case 
obtainable  by  the  use  of  the  optimum  threshold  line  angle  value,  although  the  latter  bound  has  not  been  ap¬ 
proached  in  any  reported  cases. 

Recent  research  shows  that  the  POF  and  BPOF  do  a  considerably  belter  job  than  a  classical  matched  filter 
when  the  noise  is  more  akin  to  what  is  actually  found  in  the  real  world,26  instead  of  the  white  Gaussian  noise 
associated  with  the  matched  filter.  Figure  7(a)  shows  a  tank  in  a  cluttered  desert  background;  Figure  7(b)  shows 
the  correlation  response  for  a  matched  filter  made  from  just  the  tank  (without  background),  and  7(c)  and  (d) 
show  the  responses  using  phase-only  and  binary  phase -only  filters.  The  matched  filter  shows  no  clear  correlation 
peak  at  the  on-axis  position  where  it  should  be  located.  The  other  two  filters  clearly  show  the  correlation  peak 
exactly  where  the  object  is  located. 

In  comparing  the  various  filters  it  must  be  remembered  that  the  POF  generally  provides  a  narrower  peak, 
higher  optical  efficiency,  and  greater  discrimination  against  other  shapes,  all  of  which  may  be  of  great  practical 
value.  The  tradeoff  between  these  performance  aspects  is  highly  application  specific. 


3.4  Amplitude-Encoded  Phase-Only  Filter 


It  is  possible  to  encode  the  BPOF  modulation  onto  an  amplitude  device  and  preserve  the  essential  features 
of  phase-only  filtering.22  To  see  how  this  is  possible,  note  that  when  the  binarized  phase  values  are  0  degrees 
and  180  degrees,  the  filter  values  arc  1  and  -1.  If  we  add  a  bias  term  and  renormalize,  these  become  1  and  0.  The 
bias  yields  an  extra  term  in  the  correlation  result: 


C(xj)  =  FT1  {F j(v)  ■  (Vi  +  Vi  ■ei<f>b)}, 


-  1/2 '  If  l  (*.?)  +  cb  (x3> * 
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where  Cf,  (jcj)  is  the  correlation  signal  obtained  with  the  true  BPOF.  Thus,  an  image  of  the  input  pattern 
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appears  superimposed  on  the  bin;  ■  phase-only  correlation  response.  If  the  signal  to  the  correlator  is  an  elec¬ 
tronic  signal,  and  the  signal  in  the  correlation  plane  is  detected  using  a  CCD  array,  the  two  could  be  subtracted 
to  improve  the  apparent  SNR  in  the  output  plane,  although  this  would  require  additional  post-processing  time. 
However,  experiments  and  simulations^  show  that  the  correlation  peak  is  typically  ten  times  stronger  (in  inten¬ 
sity)  than  any  secondary  peaks.  Amplitude-modulating  SLMs  are  more  common  than  phase-modulating  ones, 
and  this  technique  permits  their  use  for  binary  phase-only  filtering.  Media-like  photographic  film  can  also  be 
used.  Thus,  a  researcher  with  only  a  small  desktop  computer  can  fabricate  BPOF-like  filters  by  photographing  a 
plot  of  the  amplitude-encoded  binary  pattern  from  his  CRT  display  or  laser  printer. 


4.  SMART  FILTERS 

The  matched  filler  has  a  well  known  sensitivity  to  distortion  of  the  input  pattern  from  the  reference  pat¬ 
tern. For  example,  in-plane  rotations  of  several  degrees,  or  scale  changes  of  several  percent,  typically  result  in 
a  50  percent  reduction  in  the  correlation  peak  intensity.  The  sensitivity  strongly  depends  on  specific  parameters 
such  as  object  shape  and  spatial  frequency  bandpass  of  the  correlation  filter.  It  has  been  clear  that  this  distor¬ 
tion  sensitivity  severely  limits  the  practical  application  of  correlators  if  they  incorporate  only  a  fixed  single  filler, 
since  most  applications  of  interest  involve  many  possible  distortions  of  target  object  patterns.  The  application 
of  a  number  of  filters  to  a  given  input  scene,  that  is,  real-time  filtering,  seems  an  obvious  approach  and,  as  will  be 
seen,  such  system  concepts  are  under  investigation.  However,  the  motivation  also  exists  to  make  each  filter  as 
powerful  as  possible  to  minimize  the  number  of  filters  required  and  to  simplify  system  operation.  Such  filter 
formulations,  broadly  labeled  “smart  filters,”  are  a  current  topic  of  much  research  interest  and  will  be  reviewed 
here.  Only  space-invariant  filters  will  be  discussed.  They  provide  location  estimates  intrinsic  to  their  operation, 
do  not  require  centering  of  the  input  object  for  proper  operation  as  do  space -variant  types,  and  are  the  type  most 
frequently  used  in  Fourier  optical  processors. 


4.1  Problem  Definition 

The  problem  of  designing  distortion-invariant  correlation  filters  is  best  viewed  in  a  discrimination  context. 
In  any  practical  application  not  only  must  expected  distortions  of  targeted,  or  in-class,  patterns  (objects)  be 
recognized  (usually  by  virtue  of  a  high  correlation  peak  response),  but  one  or  more  types  of  noise,  or  out-of¬ 
class,  patterns  must  be  rejected  (usually  by  virtue  of  a  relatively  low  correlation  response).  The  classical  matched 
filter  is  the  optimal  solution  for  the  case  of  a  single  in-class  pattern  and  regarding  stochastic  noise  as  the  out-of- 
class  pattern  —  a  case  having  limited  practical  application. 

A  problem  with  complexity  more  typical  of  practical  applications  is  recognizing  certain  types  of  vehicles, 
regardless  of  orientation,  in  scenes  that  include  other  (nontargeted)  vehicles  and  natural  or  man-made  clutter 
such  as  trees,  bushes,  buildings,  roads,  etc.  Random  noise  also  may  be  present  in  the  input  patterns  as  a  result  of 
the  imaging  system  or  viewing  conditions.  As  will  be  discussed,  the  need  to  reject  out-of-class  patterns  places 
serious  constraints  on  designing  widened  in-class  response  (distortion  invariance). 

Many  applications  would  benefit  from  the  use  of  the  space-invariant  feature  of  Fourier-based  correlation  to 
furnish  a  location  estimate  of  each  recognized  target,  based  on  correlation  peak  location  in  the  output  plane. 
This  involves  additional  trade-offs  in  filter  design  since,  in  general,  optimization  of  peak  localization  (that  is, 
achieving  narrow  peaks)  is  inconsistent  with  maximizing  classification  accuracy  (for  example,  achieving  highest 


14 


signal-to-noise). 

Performance  evaluation,  and  thus  optimization,  of  correlation  filters  requires  a  systematic  set  of  perform¬ 
ance  metrics.  Unfoi  innately,  universally-accepted,  standardized  metrics  do  not  exist,  and  indeed  it  may  be 
argued  that  a  single  set  of  suen  metrics  is  inappropriate  due  to  differing  requirements  of  various  applications. 
Nevertheless,  three  broad  areas  of  performance  measurement  usually  important  to  most  applications  can  be 
identified.  These  are  discrimination,  optical  efficiency,  and  peak  localization. 

Discrimination  includes  the  concepts  normally  associated  with  terms  such  as  signal-to-noisc  (SNR)  and 
peak-to-clutter),  that  is,  the  ratio  of  target,  or  in-class,  correlation  peaks  to  responses  associated  with  non-target, 
or  out-of-elass,  inputs.  A  common  definition  of  SNR  uses  some  measure  of  correlation  peak  energy  divided  by 
the  RMS  energy  computed  over  the  correlation  plane.  An  alternate  concept,  frequently  used  in  defining 
pcak-to-cluttcr,  is  to  divide  by  the  intensity  of  the  highest  non-target  output  peak,  yielding  a  ratio  sometimes 
called  “pcak-to-sccondary.”  In  any  case,  issues  involving  the  specification  of  the  effective  size  of  the  desired 
correlation  peak  and  the  effective  energy-integration  area  (for  example,  detector  resolution)  must  be  addressed. 

Horner  efficiency  measures  the  fraction  of  processor  beam  power  concentrated  in  useful  correlation  peaks 
at  the  output  plane. 11, 24 

Peak  localization  addresses  those  properties  of  correlation  peak  shape  that  impact  the  accuracy  and  degree 
of  difficulty  of  forming  location  estimates,  essentially  peak  width  and  sidclobcs.  This  area  or  pcrfoi  mance 
measurement  is  the  poorest-defined  of  the  three  discussed  here.  Concepts  used  for  the  characterization  of 
smoothing-window  performance  in  spectrum  estimation  using  the  Fourier  transform  would  seem  to  have  ap¬ 
plicability;29  for  example,  the  6  dB  (10  dB)  width  (including  any  sidelobes)  of  a  correlation  peak  might  be  a 
reasonable  metric.  Another  approach,  illustrated  by  Kallman,-^0  is  to  chose  an  output  region  size  determined  by 
application  parameters  (for  example,  required  location  accuracy  and/or  detector  resolution)  and  count  any 
response  within  the  region  as  a  valid  correlation  peak  component  and  all  others  as  nontarget  responses.  The 
latter  approach  has  appeal  because  it  corresponds  well  to  the  actual  situation  that  arises  when  simple  threshold¬ 
ing  of  detector  array  outputs  is  used  to  define  correlation  peaks. 

The  general  problem  of  smart  filter  formulation  may  now  be  stated:  Specify  the  modulation  values  to  be 
encoded  in  each  resolvable  or  addressable  element  of  a  correlation  filter  to  optimize  correlation  performance  for 
a  given  correlation  (discrimination)  problem.  We  note  that  the  number  of  free  parameters  involved  is  deter¬ 
mined  by  the  space-bandwidth  capacity  of  the  filter  medium  and,  in  general,  can  be  quite  large,  for  example, 
16,384  for  a  128  x  128-elcment  filter.  The  specification  of  the  desired  correlation  performance  also  can  be  quite 
complex,  involving  simultaneous  optimization  of  several  performance  metrics  over  a  robust  set  of  input  scenes. 
Considerable  judgment  will  probably  be  required  just  to  formulate  an  overall  metric  that  properly  reflects  the 
tradeoffs  between  individual  performance  metrics  such  as  discrimination,  efficiency,  and  peak  localization,  and 
incorporates  a  discrimination  performance  metric  appropriately  weighted  or  averaged  over  the  input  domain. 

An  additional  complication  is  the  constraint  of  listed  ranges  or  discrete  values  of  modulation  imposed  by  certain 
filter  types  such  as  POFs  and  BPOFs.  The  nonlinear  relationships  of  these  filter  functions  to  the  Fourier  trans¬ 
form  upon  which  they  are  based  precludes  the  straightforward  application  of  many  smart  filter  formulations 
developed  for  continuous  complex  filter  modulation.  Yet  formulations  optimized  for  limited  encoding  ranges 
arc  of  practical  importance  due  to  the  limitations  of  spatial  light  modulator  devices  needed  to  implement  filters 
in  real  time. 

Thus,  the  general  smart  filter  design  problem  is  seen  to  be  both  formidably  complex  anu  very  case-specific, 
that  is,  the  optimum  filter  depends  on  the  particular  correlation  problem  and  upon  designer  judgment  regarding 
performance  tradeoffs.  It  is  not  surprising  that  no  definitive  analytical  theory  of  smart  filter  design  exists. 
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However,  effective  smart  filter  formulations  have  been  developed  and  notable  instances  of  these  will  be  reviewed 
here. 


4.2  Smart  Filters  For  Continuous* Valued  Complex  Modulation 


First  we  discuss  smart  filter  formulations  designed  with  no  restrictions  imposed  on  their  modulation 
values.  The  full  range  of  complex  numbers  is  assumed  available  in  a  SLM  or  hologram  for  physical  realization  of 
the  filter  function.  Filter  formulations  designed  for  implementation  with  limited  modulation  ranges  (for  exam¬ 
ple,  binary  phase  modulation)  will  be  discussed  in  a  following  section. 

4.2.1  COMPOSITE  FILTERS  AND  LINEAR  DISCRIMINANT  THEORY 

A  concept  underlying  many  space-invariant  smart  filter  formulations  is  the  composite  filter^1  defined  by  an 
impulse  response  that  is  a  weighted  sum  of  image  components,  frequently  a  training  set.  (Whether  the  sum  is 
performed  in  image  or  Fourier,  or  filter,  space  is  irrelevant  since  transforms  are  linearly  related  to  images.)  The 
intuitive  concept  of  a  simple  composite  filter  for  distortion-invariant  object  recognition  is  to  superimpose  dif¬ 
ferent  views  of  the  object  in  the  filter  so  eacn  view  will  have  an  equal  correlation  response  —  in  its  simplest  form 
an  average  filter.  The  concept  is  valid  although  not  a  panacea,  as  will  be  seen.  Equal  correlation  response  to 
each  view  requires  a  more  sophisticated  method  of  weighting  the  superimposed  components  than  the  equal 
weighting  implied  above,  because  they  are  not  orthogonal.  This  was  first  addressed  by  Caulfield  and  Maloney^1 
and  later  in  the  synthetic  discriminant  function  filter  formulation,  to  be  discussed. 

Composite  filter  formulations  can  be  used  to  illustrate  a  tradeoff  between  distortion-invariance  (or  widened 
in-class  response)  and  discrimination  for  correlation  filters  which  appears  to  be  fundamental,  although  subject 
to  optimization  to  varying  degrees  with  smart  filter  techniques.  Kumar  and  Popchapsky32  analyzed  and  studied 
this  tradeoff  for  composite  filters  designed  to  recognize  in-plane  rotations  and  scale  changes  of  patterns  in  the 
presence  of  additive  random  noise.  Figure  8,  taken  from  that  work,  illustrates  that  SNR  (that  is,  discrimination 
against  the  out-of-class  pattern  called  noise)  degraded  by  7  dB  as  the  number  of  superimposed  filter  components 
was  increased  from  1  to  72  to  make  the  filler  rotationally  invariant  (that  is,  widen  the  in-class  response).  Note 
that  the  SNR  degradation  is  close  to  the  number  of  components  used,  that  is,  101og(72)  =  18.6  dB.  This  example 
clearly  illustrates  the  basic  challenge  of  smart  filter  design. 

Another  important  concept  of  composite  filters,  interpolation  between  training  set  components,  also  is 
illustrated  by  Figure  8.  Note  that  when  the  training  set  components  are  spaced  by  30  degrees  (12  training  set 
components),  there  is  a  large  droop  in  correlation  response  for  input  rotations  between  the  training  set  angles. 
Training  with  five-degree  increments  appears  necessary  to  achieve  good  interpolation.  The  filter  designer  must 
choose  a  training  set  that  ensures  good  interpolation  between  components.  An  intuitive  approach  is  to  base  the 
interval  between  components  on  the  distortion  sensitivity  exhibited  by  a  single-view  filter,  which  seems  to  apply 
to  the  case  shown  in  Figure  8.  A  broader  perspective  of  the  interpolation  issue  is  that  a  training  set  must  be 
statistically  representative  of  its  class.  This  is  expected  to  be  a  significant  problem  in  many  cases  because  of  the 
high  dimensionality  of  the  images  being  processed  relative  to  that  of  typical  composite  filters  (which  is  on  the 
order  of,  or  equal  to,  the  number  of  training  set  components).  The  composite  filter  approach  may  be  viewed  as  a 
method  of  parametrizing  filter  design  which  dramatically  reduces  the  dimensionality  of  the  problem  (from  the 
space-bandwidth  of  the  filter)  but  involves  attendant  limitations  including  the  interpolation  problem. 
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Figure  8.  Analytical  results  indicating  the  intcrpolation-vs-discrimination  tradeoff  of  composite  filters.  Correlation 
signal-to-noisc  vs  rotation  angle  of  input  pattern  for  filters  computed  usingvariou,  training-set  intervals,  taken  from 
Kumar  and  Popchapsky  (Ref.  32). 


The  on-axis  correlation  value,  (the  amplitude  value  found  at  the  center  of  the  correlation  plane  for  a  cen¬ 
tered  input  pattern),  is  the  functional  inner  product  of  the  unshifted  input  and  reference  patterns,  which  may  be 
viewed  either  as  two-dimensional-or  one-dimensional-  (sequentially-scanned)  functions  or  (when  sampled) 
vectors.  Thus,  the  correlation  (with  respect  to  the  on-axis  result  only)  implements  a  linear  discriminant  func¬ 
tion,  w  hich  has  led  to  a  large  body  of  work3-3'37  concerning  optical  implementation  of  linear  discrimination 
concepts.3**  The  performance  of  such  classifiers  can  be  limited  in  some  applications  by  their  reliance  on  only  the 
on-axis  correlation  response.  In-class  distortions  or  out-of-class  inputs  not  represented  in  the  filter  can  yield 
other  peaks  greater  than  the  on-axis  value.  These  arc  confused  with  valid  peaks  since,  in  general,  the  post¬ 
processor  does  not  know  the  location  of  inputs  and,  in  fact,  is  usually  relying  on  the  correlation  process  to  fur¬ 
nish  a  location  estimate,  as  mentioned. 

4.2.2  SYNTHETIC  DISCRIMINANT  FUNCTION  FILTERS 

The  first  widely  used,  and  probably  best  known,  smart  filter  was  the  synthetic  discriminant  function  (SDF) 
filtcr,33although  several  closely  related  approaches  previously  had  been  reported.31  The  SDF  filter  is  a  compos¬ 
ite  filter  in  which  the  weights  are  set,  using  linear  discrimination  techniques,  to  yield  specified  on-axis  correla¬ 
tion  responses  over  the  in-class  and  out-of-class  training  sets.  In  its  first  form33  called  the  cqual-correlation- 
pcak  (ECP)  SDF,  the  on-axis  covariance  matrix  for  the  training  set  is  foi  med  and  linear  algebra  techniques  arc- 
applied  to  solve  the  composite  filler  weights  which  ensure  equal-corrclation-pcak  response  over  the  in-class 
training  set.  Problems  with  (spurious)  correlation  responses  larger  than  the  (desired)  on-axis  response  were 
addressed  by  the  correlation-SDF  (c-SDF)  variant  which  incorporates  control  of  sidclobc  responses  not  included 
in  the  equal-corrclation-peak  formulation.39 

Variants  of  the  SDF  have  been  developed,  several  of  which  arc  called  optimal  linear  discriminant  function 
(OLDF)  filters.-35  Notable  recent  developments  include  the  minimum  average  correlation  energy  (MACE) 
SDF‘}()  and  the  minimum-variance  SDF  (M  VSDF).^1  A  generalized  SDF  formulation,  encompassing  most  of 
the  previous  types  as  special  cases,  has  been  reported  recently.^ 

SDF  filters  have  demonstrated  good  performance  both  in  simulations39"11*  ‘*3"17  and  experiments.^* 9 ' 
Except  for  the  cSDF,  performance  of  these  analytical  SDF  formulations  is  limited  by  their  control  of  only  the 
on-axis  correlation  response  and  that  no  phase  variation  of  that  response  is  allowed,  although  the  detected 
output  of  the  correlator  is  intensity,  which  is  not  sensitive  to  phase.  Also,  a  practical  disadvantage  is  that 
straightforward  application  of  minimum-variance  SDF  and  gcneraliz.cd-SDF  formulations  requires  linear  alge¬ 
bra  operations  on  huge  matrixes  (N-squarcd  on  a  side  for  N  X  N  images),  since  the  cigcnsolution  of  the  raw 
image  space  is  part  of  these  formulations.  However,  the  SDF  filter  variants  probably  comprise  the  best  de¬ 
veloped  and  most  powerful  analytical  smart  filler  formulations. 

Generalized  matched  filters37  also  are  based  on  linear  discrimination  concepts,  applied  in  this  ease  in  the 
Fourier  (filter)  domain.  The  goal  of  the  filter  formulation  is  a  (complex)  linear  discriminant  function  (the  filter) 
that  best  separates  the  target  and  nontarget  classes.  Rigorous  application  of  the  formalism  involves  eigensolu- 
tion  of  huge  matrixes,  but  with  a  simplifying  assumption  (that  different  Fourier  components  are  uncorrclatcd 
over  a  class)  tractable  solutions  result.  Simulations  using  the  simplified  formalism  have  yielded  good  results  in  a 
case  in  which  nine  distortions  of  an  aircraft  silhouette  were  to  be  discriminated  from  zero-mean  Gaussian 
noise.-37  This  formalism  also  has  the  limitation  of  controlling  only  the  on-axis  correlation  value. 

Kallman  has  reported  SDFs  formulated  by  iterative  numerical  techniques  using  min-max  performance 
criteria  in  which  complex  weighting  coefficients  were  allowed  and  response  was  controlled  over  the  entire  output 
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planed  48  jn  test  simulations,  these  filters  performed  substantially  better  than  cqual-corrclation-peak  SDF 
filters  for  the  same  inputs,  as  illustrated  in  Figure  9.  Part  (a)  is  a  typical  input  scene  from  these  simulations, 48 
consisting  a  target  tank  embedded  in  clutter  composed  of  a  “junkyard  of  tank  parts."  Figure  9  (b)  and  (c)  are 
profiles  of  the  correlation  planes  for  the  equal-correlation-pcak  and  Kallman-rccipe  SDF  filters,  demonstrating 
substantial  improvement  in  peak-to-clutter. 

The  iterative-search  filter  formulation  algorithm  used  by  Kallman  does  not  require  operations  on  huge 
matrixes,  but  is  computationally  intensive  in  its  own  right,  in  part  because  each  point  in  the  correlation  plane  is 
examined  and  controlled.  The  use  of  complex  weighting  coefficients  doubles  the  dimensionality  of  the  filter 
parametrization  over  that  of  the  cqual-corrclation-peak  SDF  filter  formulations  using  real  weights.  Although 
not  expressible  in  closed  form,  and  thus  not  amenable  to  general  analytical  performance  assessment,  the  Kall¬ 
man  SDF  approach  uses  a  filter  performance  metric  that  is  more  appropriate  for  many  applications,  and  it  has 
the  potential  to  yield  superior  filters  as  a  result  of  the  additional  free  parameters  associated  with  complex 
weights.  The  Kallman  recipe  is  an  example  of  a  general  class  of  filter  design  approaches  involving  parametriza- 
tion  of  the  filter  (to  reduce  dimensionality)  and  iterative  search  over  the  parameter  space  to  find  a  (probably 
local)  maximum  of  a  performance  metric.  Such  approaches  presumably  could  benefit  from  global  search  con¬ 
cepts  such  as  simulated  thermal  ••nncaling4^  which  are  purported  to  find  performance  maxima  near  the  global 
maximum.  A  simulation  demonstrating  the  use  of  this  approach  to  optimize  discrimination  between  two  charac¬ 
ters  in  a  BPOF  filter  has  been  reported.^ 

4.2.3  CIRCULAR  HARMONIC  FILTERS 

Filters  based  on  circular  harmonic  decompositions  are  naturally  suggested  when  invariance  to  in-plane 
rotation  is  desired,  and  they  have  been  the  basis  for  a  substantial  body  of  research. ^-55  ^  fundamental  limita¬ 
tion  of  circular  harmonic  composite  filters  is  that  only  in-plane  rotation  invariant  response  is  obtained,  and  only 
if  a  single  circular  harmonic  number  is  incorporated  in  the  filler.^1-  ^  This  comprises  a  serious  limitation  lor 
many  practical  cases,  since  a  single  harmonic  may  contain  only  a  small  fraction  of  the  shape  information  critical 
to  recognition  and  discrimination.  One  attractive  approach  for  pattern  recognition  using  circular  harmonics 
involves  forming  a  feature  vector  based  on  the  responses  of  3  bank  of  filters,  each  based  on  a  different  circular 
harmonic  number.-*’'-'-'  T  he  resulting  feature  vector  is  rotationally  invariant  and  can  embody  robust  informa¬ 
tion  related  to  object  shape,  supporting  good  recognition  and  discrimination. 

Although  a  given  circular  harmonic  always  provides  rotation-invariant  correlation  intensity  response,  the 
distribution  of  correlation  responses  over  harmonic  number  depends  on  the  ccnter-of-expansion  chosen  for  the 
decomposition.  Excessive  sidelobes  can  result  if  the  proper  center  is  not  used. 

Fouricr-Mcllin  descriptors,  comprising  radial  moments  of  circular  harmonic  filters,  have  been  investigated 
for  scale-  and  rotation-invariant  pattern  recognition.^ 

Circular  harmonic  smart  filter  techniques  are  particularly  attractive  for  applications  where  in-plane  rota¬ 
tion  is  the  predominant  form  of  in-class  distortion. 

4.2.4  MELLIN  RADIAL  HARMONIC  FILTER 

Mcndlovic  ct  al^6  have  reported  a  titter  based  on  the  decomposition  of  a  function  in  terms  of  azimuthally 
weighted  radial  functions  (called  Mellin  Radial  Harmonics)  that  exhibit  radial  scale  invariance.  This  formalism 
is  a  dual  of  the  circular  harmonic  filter  and  provides  filters  that  are  invariant  to  in-plane  scale  changes,  provided 
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F  igure  «>a.  Example  olSDF-  filler  performance  achieved  by  Kallman  (Ref.  4X|.  Input  scene:  tank  in  clutter  described 
as  "a  junkyard  of  tank  parts." 


Figure  Vb  Flxample  of  SDF  i i Iter  performance  achieved  by  Kallman  |Rcf.  4S|.  Correlation  profile  using  usual  SDF 
formulation 


Figure  9c.  Example  of  SDF  filler  performance  achieved  by  Kallman  |Rcf.  48).  Correlation  profile  using  Kallman 
formulation. 


only  a  single  Mellin  harmonic  is  embodied  in  the  filter.  An  analogous  dependence  on  proper  choice  of 
center-of-expansion  also  exists.  Experimental  correlations  demonstrating  recognition  and  discrimination  of  the 
letter  *E’  over  a  4:1  range  of  scale  were  reported.^6 

4.2.5  SPATIAL-TEMPORAL  CORRELATION  FILTER 

The  (space-invariant)  spatial-temporal  correlation  (STC)  filter^7  formulation  is  based  on  the  polar-log 
coordinate  transform,  which  was  the  basis  of  a  well-known  spacevariant  correlation  technique  possessing  both 
in-plane  scale  and  rotation  invariance.^^  The  method  constructs  a  filter  to  recognize  one  object  and  defines  a 
signature  control  function  (SCF)  that  specifies  the  variation  of  correlation  peak  amplitude  over  360  degrees  of 
rotation  and  a  range  of  scale  change.  If  the  singature  control  function  is  set  constant  over  the  distortion  control 
region,  an  equal-corrclation-pcak  response  filter  is  produced,  but  in  principle  any  variation  of  peak  response  as  a 
function  of  distortion  variables  may  be  programmed. 

A  particular  case  demonstrated  in  reported  examples  involves  programming  for  uniform  response  over  the 
scale  distortion  range  and  a  sharply  peaked  response  in  rotation  angle.  Thus,  if  the  input  is  rotated,  a  temporal 
signature  which  peaks  at  a  time  determined  by  the  rotation  angle  of  the  target  object  is  produced  at  the  correla¬ 
tion  plane.^7  Simulations  using  computer-simulated  aircraft  outlines  demonstrated  the  expected  behavior  for 
this  type  filter.^7  The  programmed  temporal  signature  was  obtained  only  at  an  output  location  corresponding 
to  the  location  of  the  target  aircraft  in  the  input,  and  was  obtained  for  differently-scaled  versions  of  the  target. 

In  other  simulations,^7  spatial-temporal  correlation  filters  for  estimating  the  rotation  angle  of  airplane  outlines 
based  on  correlation  peak  value  were  constructed. 

The  spatial-temporal  correlation  filter  controls  only  the  on-axis  correlation  peak  response  for  a  single 
target  pattern,  and  addresses  only  in-plane  distortions.  It  uses  no  training  set,  but  addresses  distortion  in  a 
continuous  fashion.  The  filter  has  a  closed-form  mathematical  expression  and  its  formulation  is  straightforward, 
involving  log-polar  coordinate  transformations  and  discrete  Fourier  transforms. 

4.2.6  THE  LOCK-AND-TUMBLER  FILTER 

The  lock-and-tumblcr  (LAT)  filter60'62  is  a  composite  filter  based  on  a  superposition  of  cigcnimagcs  of  a 
training  set  representing  any  range  of  any  type  of  distortion.  A  spectral  iteration  technique  is  used  to  find  the 
(complex)  weights  that  solve  the  design  constraint  of  the  filter,  which  is  that  the  on-axis  intensity  be  constant 
over  a  bank  of  LAT  filters  for  a  given  training-set  input.  Typically  100  or  more  cigenimage  components  are 
included  in  each  filter.  In  the  special  cases  where  only  in-plane  rotation  or  scale  distortions  are  addressed,  the 
cigcnimagcs  are  circular  harmonics  or  Mellin  components  respectively. 

The  use  of  the  LAT  filter  is  fundamentally  different  from  all  other  filters  in  that  the  correlation  output 
pattern  must  be  post -processed  to  find  the  point(s)  exhibiting  minimum  variance  taken  over  a  number  (typically 
20)  of  input-filter  correlation  pairs.  Recognition  and  location  of  a  target  is  based  on  existence  of  a  peak  in  the 
post -processed  pattern  of  mean-lo-varianee  ratio  calculated  for  each  pixel.  Simulations60- 62  and  experiments61 
have  verified  the  feasibility  of  the  filter. 

The  bank  of  LAT  filters  is  distinct  only  by  virtue  of  the  stariing-sced  of  expansion  coefficients  used  in  the 
spectral  iteration.  A  multiplicity  of  filters  satisfying  the  identical  design  constraint  is  possible  since  the  number 
of  spectral  weights  greatly  exceeds  the  number  of  constraints  applied  in  the  filter  solution.  Eigensolution  in¬ 
volves  linear  algebra  operations  on  M  X  M  matrixes,  where  M  is  the  number  of  training  set  images.  To  minimize 
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computations  but  retain  sufficient  interpolation,  the  training  set  must  be  carefully  chosen,  and  this  is  done  with 
regard  to  the  cross-correlation  coefficients  of  the  set.^2  In  its  first  reported  form,^  the  I.AT  involved  circular 
harmonic  eigenimages  and  the  equivalent  of  a  bank  of  N  LAT  filters  could  be  obtained  by  taking  N  different 
relative  rotations  of  the  input  and  a  single  filter. 

The  LAT  has  been  applied  successfully  in  simulations  to  provide  perspective-invariant  recognition  of  a 
model  tank,  using  captured,  median  filtered,  and  edge-enhanced  images,  over  360  degrees  of  rotation  and  60 
degrees  of  zenith-angle  variation .62  in  this  case  380  training  set  images  and  an  equal  number  of  eigenimages 
were  used.  Interpolation  and  discrimination  of  other  shapes  was  demonstrated  using  20  LAT  fillers.  Figure  1(1 
is  an  example  taken  from  these  simulations.  Figure  10(a)  is  an  input  scene  containing  three  different  perspective 
views  of  the  target  tank  and  one  nontarget  vehicle  image.  Figure  10(b)  is  a  processed  output-pattern  plot  indi¬ 
cating  recognition,  with  good  pcak-to-cluttcr  ratio,  of  the  three  targets  and  rejection  of  the  nontarget  .62 

LAT  filter  formulation  is  complicated  and  computationally-intensive,  but  its  use  of  eigenimages  to  ensure 
incorporation  of  all  training  set  information  is  well  founded  and  the  performance  exhibited  in  the  perspective- 
invariant  recognition  problem  just  discussed  is  impressive.  LAT  issues  requiring  attention  include  the  effects  of 
clutter,  nonzero  backgrounds,  and  nonuniform  illumination,  and  real-time  implementations.  (The  filter  is 
complex  valued,  hence  no  SLM  can  write  it  directly.) 

4.2.7  SUMMARY  OF  STATUS  OF  SMART  FILTERS  FOR 
CONTINUOUS-VALUED  ENCODING 

Our  discussion  has  indicated  that  a  diverse  selection  of  smart  filter  techniques  is  evolving.  They  differ  in 
mathematical  approach  from  simple  ad  hoc  concepts  to  sophisticated  treatments;  in  solution  method  from 
closed  form  to  iterative  searches;  and  in  goals  from  controlling  only  peak  response  to  controlling  responses  over 
the  entire  correlation  output  plane.  Some  use  training  sets;  others  do  not.  As  stated,  no  general  all-purpose 
formulation  has  emerged,  and  such  a  development  would  be  surprising  in  view  of  the  complex  and  sometimes 
conflicting  demands  that  apply. 

Correlation  (discrimination)  problems  also  span  a  diverse  range.  They  differ  markedly  in  complexity  and 
goals,  and  some  readily  furnish  training  sets  while  others  may  not.  Thus,  it  seems  fitting  that  a  wide  spectrum  of 
smart  filter  design  tools  is  being  developed.  All  of  the  filter  formulations  have  shown  good  performance,  at  least 
in  computer  simulations,  applied  to  specific  problems.  With  the  possible  exception  of  the  equal  correlation 
peak  SDF  filter,  none  have  been  tested  sufficiently,  cither  experimentally  or  by  simulation,  to  establish  a  basis 
for  definitive  assessment  of  their  overall  capability. 

Experimental  implementation  of  the  filters  discussed  in  this  section  has  lagged  theory  and  simulations,  both 
because  simulations  are  a  logical  and  easier  first  step,  and  because  spatial  light  modulator  technology  does  not 
readily  support  the  real  time  implementation  of  continuous  complex-valued  filter  functions.  Continued  research 
and  development  is  expected  to  attack  both  these  limitations. 

4.3  Smart  Filters  for  Limited  Modulation  Range  Encoding 

In  this  section  we  discuss  smart  filter  formulations  for  implementation  with  limited  modulation  ranges,  in 
particular  for  POF,  BPOF,  and  ternary  phase-amplitude  filter  (TPAF)  encoding.  The  TPAF  is  a  more  recent 
development  and  will  be  described.  As  mentioned,  the  motivation  for  such  filters  is  practical,  related  primarily 
to  the  necessity  of  implementation  with  available  SLMs  that  only  modulate  cither  phase  or  amplitude,  with  only 
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Figure  10.  Example  of  Lock-and-tumblcr  filler  performance  [Ref.  62].  (a)  Input  scene 
containing  three  views  of  target  tank  (two  of  which  were  not  in  training  set)  and  one  nontar¬ 
get  (bulldozer),  (b)  Output  plane  showing  peaks  corresponding  to  three  views  of  target  tank. 
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binary  or  ternary  (three-level)  modulation  available  in  some  cases,  and  to  reduce  on-line  storage  requirements. 

The  fundamental  smart  filter  design  problem  has  been  defined  and  is  the  same  here  except  that  the  addi¬ 
tional  constraint  of  the  limited  encoding  range  must  be  treated.  As  mentioned,  many  filter  design  formalisms 
arc  profoundly  impacted  by  this  constraint.  Processing  a  filter  function  to  yield  the  required  encoding,  (for 
example,  taking  only  the  phase,  or  thresholding  to  define  binary  phase),  is  a  nonlinear  operation  and  produces  a 
totally  new  filter  function  not  necessarily  satisfying  the  design  goals  of  the  original  continuous-valued  filter  func¬ 
tion.  Many  formalisms,  including  the  original  equal-correlation-peak  SDF  and  LAT  filters,  cannot  readily 
encompass  the  required  nonlinear  step,  and  thus  have  not  been  revised  for  limited-range  encoding. 

4.3.1.  CONVERTED  CONTINUOUS-VALUED  COMPLEX  FILTERS 

The  phase-only  concept  is  a  very  general  principle  that  can  be  applied  to  any  complex  filter  function  by 
merely  retaining  the  phase  (or  binarizing  it)  and  setting  the  modulus  to  unity,  as  first  proposed  by  Horner  and 
Gianino  for  the  p-SDF  filter.^4 

Researchers  first  created  smart  POFs  and  BPOFs  for  testing  by  simply  converting  continuous-valued 
complex  smart  filters  of  the  equal-correlation-peak  SDF,  Kallman-SDF,  or  GMF  types 24.  45, 63  was 
recognized  that  this  was  sub-optimal,  for  the  reasons  stated  above,  but  the  simplicity  of  the  approach  was  appeal¬ 
ing  and,  initially,  no  filters  specifically  optimized  for  these  encoding  methods  existed.  Results  were  mixed.  In 
some  cases  reasonably  good  filters  resulted,  although  they  did  not  provide  the  performance  (for  example,  equal 
correlation  peaks  over  the  in-class  set)  designed  into  the  equal-correlation-peak  SDF. 

4.3.2  DIRECTLY  OPTIMIZED  POF  AND  BPOF  FILTERS 

Smart  filters  optimized  for  phase-only  and  binary  phase-only  encoding  over  in-class  and  out-of-class  train¬ 
ing  sets  were  reported  by  Kallman64  and  by  Jared  and  Ennis, 66  using  distinctly  different  approaches. 

Kallman  directly  optimized  the  N-  phase  values  of  a  NX N  POF  using  iterative  search  techniques  and  his 
min-max  performance  criteria.64  BPOF  filters  were  created  from  the  resulting  POF  patterns  after  searching  for 
an  optimum  threshold  line  angle ,  and  thus,  were  not  directly  optimized  for  that  encoding.  These  POF  and 
BPOF  filters  provided  significantly  better  discrimination  performance  than  did  converted  SDFs  for  out-of-plane 
distortions  of  military  vehicles  in  random  backgrounds.6^  The  filters  also  exhibited  interpolation  between  train¬ 
ing  set  components.  The  numerical  methods  used  in  the  formulation  of  these  filters  involve  considerable 
computation  but  they  address  meaningful  optimization  criteria  and  provide  good  performance. 

Jared  and  Ennis6^  developed  an  ad  hoc  iterative  technique,  called  the  fillcr-SDF  (fSDF)  which  adjusts  the 
weights  of  a  composite  transform,  based  on  training  set  images,  so  as  to  control  the  on-axis  correlation  response 
of  the  converted  POF  or  BPOF  over  the  training  set,  that  is,  addressing  the  goals  of  the  cqual-correlation-pcak 
SDF  filter.  The  formulation  process  is  numerically  relatively  simple,  but  does  not  converge  for  some  cases.  The 
use  of  only  real  weights  and  control  of  only  the  on-axis  peaks  are  additional  limitations  of  the  approach  as  re¬ 
ported,  just  as  for  the  original  SDF  filter.  Impressive  simulation  results  for  a  limited  set  of  test  images  have  been 
reported.66  If  the  above  limitations  are  acceptable,  and  convergence  can  be  obtained  for  the  problem  at  hand, 
this  technique  provides  a  relatively  simple  method  to  construct  POF  and  BPOF  filters  of  the  SDF  type. 
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4.3.3  CIRCULAR  HARMONIC  PHASE-ONLY  COMPOSITE  FILTERS 


Circular  harmonic  phase-only  filters  have  recently  been  reported.67-  ^  As  with  continuous-valued  com¬ 
plex  modulation,  composite  circular  harmonic  phase-only  filters  must  be  restricted  to  one  harmonic  to  retain 
rotation  invariance.  Such  filters  have  demonstrated  performance  superior  to  continuous-valued  versions  for  test 
cases  involving  a  small  set  of  printed  characters6^  and  images  of  the  space  shuttle  on  a  background.67  Even 
better  performance  was  obtained  when  some  amplitude  information  derived  from  the  nontarget  letter  was 
incorporated  in  the  filter,  but  such  a  filter  is  no  longer  phase-only. 

The  behavior  of  phase -only  circular  harmonic  filters  relative  to  continuous-valued  versions  is  as  might  be 
predicted  based  on  the  comparison  of  phase-only  matched  filters  to  their  continuous  complex  counterparts. 
Narrower  peaks,  greater  discrimination,  and  much  higher  Horner  efficiencies  are  exhibited  by  the  continuous 
complex  filters.  Basic  characteristics  of  circular  harmonic  filters,  including  in-plane  rotation  invariance  (for  a 
single  harmonic)  and  sensitivity  to  ccnter-of-expansion  are  retained. 

4.3.4  TPAF  SM/RT  FILTERS 

Smart  filter  formulations  for  ternary  phase-amplitude  encoding  (-1,0, 1  modulation  levels)  have  been 
reported.0^'  7^  This  work  treated  the  filter  as  a  product  of  binary  amplitude  (0, 1  modulation)  and  binary  phase 
(-1,1  modulation)  patterns  that  were  independently  formulated,  that  is,  a  “two-bit”  filter.  This  concept  goes 
beyond  simple  high-pass  and  low-pass  filter  cutoff  concepts  in  that  each  filter  pixel  can  be  turned  off  (set  to  zero 
modulation)  in  a  pattern  designed  to  optimize  the  filter’s  performance.  Ternary  filter  patterns  also  were  investi¬ 
gated  incidentally  in  work  previously  discussed  in  the  context  of  BPOF  formulation.  The  Kallman  BPOFs64 
included  a  low  frequency  box  of  zero-modulation  (DC  block)  chosen  by  trial-and-error  to  optimize  filter  per¬ 
formance.  Casascnt  and  Rozzi4^  investigated  SDF  filters  converted  to  TPAF  modulation  with  the  amplitude 
pattern  determined  by  thresholding  on  the  magnitude  of  an  SDF  filter  function.  The  use  of  bandpass  filtering 
has  been  shown  to  optimize  SNR  for  simple  POFs  and  BPOFs, 21  and  in  the  latter  case,  a  TPAF  is  created. 

Here,  we  arc  concerned  with  formulations  making  more  sophisticated  use  of  the  amplitude  modulation  capabili¬ 
ty  to  optimize  performance,  though  we  note  that  choice  of  an  optimum  processor  overall  bandpass  should  always 
be  considered. 

The  primary  benefit  expected  of  TPAF  modulation,  by  virtue  of  the  zero-modulation  state  added  to  the 
BPOF  modulation  values,  is  improved  discrimination.  This  is  believed  to  result  from  selective  blocking  of  spa¬ 
tial  frequencies  associated  with  out-of-class  patterns.  This  concept  was  the  basis  of  the  transform-ratio  (TR) 
technique  for  setting  the  zero-modulation  state  (or  equivalently,  setting  the  binary  amplitude  portion).7^  The 
basic  idea  of  the  TR  method  is  to  form,  in  some  appropriate  manner,  representative  in-class  and  out-of-class 
power  spectral  density  patterns.  The  two  patterns  are  compared,  point-by-point,  and  the  binary-amplitude 
pattern  is  established  by  thresholding  on  the  ratio  of  in-class  to  out-of-class  values.  It  is  of  practical  importance 
to  treat  the  TR  threshold  as  a  design  parameter  of  the  filter,  varying  it  to  search  for  optimum  performance.  The 
phase-only  portion  of  the  filter  can  be  set  according  to  any  method  normally  used  for  BPOF  formulation,  al¬ 
though  it  would  seem  desirable  to  address  distortion  invariance  (widened  in-class  response)  with  the  binary 
phase  pattern,  since  the  primary  purpose  of  the  binary  amplitude  pattern  is  to  improve  discrimination.  Simula¬ 
tions  have  demonstrated  the  expected  discrimination  improvements  (relative  to  a  simple  BPOF)  against  both 
noise  and  other  objects.7^ 

The  filtcr-TPAF  (fTPAF)  formulation6^  combines  the  TR  method  of  setting  a  binary-amplitude  pattern 
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with  the  iterative  method  used  in  the  fSDF6^  to  form  a  binary-phase  pattern,  resulting  in  a  smart  TP AF  which 
has  controlled  distortion-invariance  and  discrimination.  Simulations  addressing  a  multi-font  character  recogni¬ 
tion  problem  have  demonstrated  the  superior  classification  performance  of  this  type  filter  compared  with  con¬ 
verted  SDF  filters  and  the  fSDF-BPOF  formulation,71  as  illustrated  in  Figure  11.  Figure  11(a)  shows  the  var¬ 
ious  in-class  and  out-of-class  characters  used  in  the  study.  Figure  1 1  (b)  and  (c)  are  bar  plots  showing  the  im¬ 
proved  discrimination  performance  of  the  fTPAF  over  the  binarized  projection  SDF  filter. 

The  fTPAF  formulation  evaluated  filler  performance  over  the  entire  correlation  output  plane  after  each 
iteration  of  filter  weights,  a  significant  departure  from  the  reported  fSDF  iteration  procedure.  Thus,  the  filters 
controlled  performance  over  the  entire  correlation  plane  rather  than  only  the  on-axis  responses.  This  was  criti¬ 
cal  to  success  for  this  application.71  As  for  the  fSDF,  the  ad  hoc  fTPAF  iterative  formulation  procedure  docs 
not  always  converge. 

TPAFs  have  exhibited  low  correlation  efficiencies  relative  to  POFs  and  BPOFs  in  reported  simulations,69, 
7tl-  71  this  is  not  surprising  since  they  typically  block  up  to  90  percent  of  the  filter  plane  with  zero-modulation 
states.  In  the  worst  cases,  the  degradation  factor  was  about  two  orders  of  magnitude.  For  proper  perspective,  it 
must  be  remembered  that  POFs  and  BPOFs  typically  exhibit  correlation  efficiencies  20-50  times  that  of  classical 
matched  filters.  Thus,  even  the  least  efficient  TPAFs  may  have  efficiencies  within  an  order  of  magnitude  of  a 
classical  matched  filter.  (This  discussion  has  ignored  practical  efficiency  factors  of  the  optical  implementation 
and  filler  medium,  which  arc  independent  of  filter  formulation.) 


4.4  Summary 

Phase-only  and  discrete-valued  filters  are  a  relatively  recent  development  in  optical  correlation,  first  re¬ 
ported  in  1984.  Thus,  the  development  of  smart  filter  techniques  specifically  optimized  for  such  filter  encodings 
is  at  an  early  stage.  However,  the  experimental  testing  of  such  smart  filters  will  be  accelerated  by  the  availability 
of  SLMs  suitable  for  their  implementation,  since  this  practical  factor  was  one  motivation  for  initially  considering 
these  filter  types. 

The  current  selection  of  smart  filter  formulations  for  limited-range  encoding  is  not  as  diverse  as  for  contin 
uous  filter  functions,  but  research  is  proceeding  and  the  filters  already  developed  have  shown  good  promise,  as 
discussed. 

Simple  forms  of  BPOF  smart  filter  already  have  been  implemented  experimentally,  demonstrating  good 
performance  and  agreement  with  computer  simulations.7^.  73 

TPAFs  having  non  trivial  zero-modulation  patterns  (that  is,  other  than  simple  low-  and  high-pass  filters) 
have  not  been  implemented  experimentally,  but  this  situation  is  likely  to  change  in  the  near  future,  since  it 
recently  has  been  shown  that  TPAF  modulation  can  be  achieved  with  relative  ease  in  magneto-optic  spatial  light 
modulators.7-1 

POF  and  discrete-valued  smart  filters  are  attractive  due  to  their  potential  for  near-term  real-time  imple¬ 
mentation  with  available  SLMs.  A  definitive  assessment  of  the  relative  performance  of  these  filter  types  is  not 
available,  but  the  reported  results  are  promising  and  investigation  of  them  is  continuing. 

5.  UKAL-TIMI.  IMI’I.KMLNTATION 


This  action  will  review  experimental  realizations  of  real-time  Fourier  optical  correlation,  limited  by  defini¬ 
tion  to  those  architectures  providing  rapid  changes  o (both  input  and  reference  patterns.  Spatial  light  modula- 
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Figure  1  la.  Example  of  fTPAF  character  recognition  simulation  results  (Ref.  71).  Charac¬ 
ters  used  in  study. 
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Training  Set 


Figure  11.  Example  of  fTPAF  character  recognition  simulation  results  [Ref.  71  j.  (h)  Dis¬ 
crimination  achieved  with  pSDF  filter,  (c)  Discrimination  achieved  with  ITPAK 


tors  (SLM)  are  the  key  element  in  such  systems.  They  will  be  mentioned  where  applicable,  but  not  reviewed 
systematically.  The  reader  is  referred  to  the  literature7**'^  and  the  paper  by  J.  Neff  and  R.  Athale  in  the  Octo¬ 
ber  1989  Proc.  IEEE  [77  (No.  2)]  for  further  information  on  the  technology  of  these  devices.  Correlator  per¬ 
formance  is  limited  by  SLM  technology,  particularly  in  the  areas  of  resolution,  frame  rate,  and  fidelity  of  modu¬ 
lation.  The  correlation  architectures  now  considered  attractive  for  practical  application  have  evolved  through 
advances  in  both  SLM  technology  and  theoretical  concepts  which  capitalize  on  available  SLM  performance. 

Optical  correlators  have  not  achieved  everyday  use  in  either  commercial  or  military  contexts,  although  their 
potential  for  both  areas  has  been  recognized  and  has  motivated  much  of  the  research  performed.  Thus,  mainly 
we  will  be  discussing  laboratory  experiments,  although  the  work  has  been  carried  into  the  breadboard  or  brass- 
board  stage  in  a  few  instances,  which  will  be  noted. 

Most  real-time  correlation  experiments  fit  into  the  two  broad  architectural  categories  already  discussed, 
frequency  plane,  and  joint-transform  correlation.  Variants  involving  phase -only,  binary,  or  ternary  encodings 
exist  in  both  categories. 

5.L  Real-Time  Matched  Filtering  Experiment 

Any  type  of  SLM  can  be  used  for  the  real-time  input  function,  with  optically  or  electrically  addressed  types 
being  selected  as  required  by  the  application.  Real-time  implementation  of  complex  continuous-valued  filter 
functions  (for  example,  matched  filters)  requires  modulation  capability  (that  is,  independent  magnitude  and 
phase  programming)  not  provided  by  any  existing  SLM,  unless  a  holographic  (carrier-frequency)  encoding  is 
used.  The  latter,  although  possible,  is  expensive  in  terms  of  required  SLM  space-bandwidth,  typically  exacting  a 
penalty  of  a  four-fold  reduction  in  information  capacity  to  establish  the  carrier  spatial  frequency,  and  an  addi¬ 
tional  factor  if  magnitude  is  encoded  in  other  than  analog  form,  as  in  CGH  schemes. 

Despite  the  difficulties  mentioned,  experiments  implementing  continuous  complex  spatial  filters  in  real 
time  have  been  reported  and  will  be  reviewed  here. 

Four-wave  mixing  in  photorefractive  crystal,  such  as  bismuth  silicon  oxide  (BSO),  can  be  used  to  form 
correlation  filters  in  real  time.77  Good  agreement  between  digitally  calculated  correlations  and  experimental 
results  were  reported,  and  correlation  rise  times  of  less  than  200  ns  were  observed  with  a  Q-switchcd  frequency- 
doubled  Nd:  YAG  source.  The  architecture  is  somewhat  akin  to  joint-transform  correlation  and  requires  input 
of  the  reference  pattern  in  the  image  (rather  than  Fourier)  domain. 

A  portable  brassboard  correlator  achieving  rapid  sequencing  between  four  fixed  holographic  filters  using  an 
array  oflascr  diode  sources  has  been  constructed  and  tested.7^  A  Hughes  liquid  crystal  light  valve  (optically 
addressed)  SLM  served  as  the  input  device. 


5.2  Real-Time  Binary  Phase-Only  and  Ternary  Phase-Amplitude  Correlator 

The  first  real-time  correlator  implementing  BPOF  filters,  reported  in  1986,  used  48  x  48  element  electrical¬ 
ly-addressed  magneto-optic  SLMs  (MOSLM)  at  both  input  and  filter  plane  79  Nontrivial  patterns  could  be 
correlated  with  this  limited  resolution  because  no  filter  encoding  capacity  was  sacrificed  to  implement  a  holo¬ 
graphic  spatial  carrier.  The  processor  is  an  in-line  configuration.  It  cycled  new  inputs  and/or  filters  at  the  rate  of 
four  per  second,  limited  by  the  controlling  microcomputer  (an  Apple  lie)  rather  than  the  SLMs. 

Further  experiments  with  this  correlator  have  demonstrated  good  correlation  performance  in  character  and 
machine  part  recognition  applications,  including  successful  implementation  of  simple  smart  filters.72  Real-time 


31 


magneto-optic  SLMS  correlators  using  128  x  128  element  devices  also  have  been  reported.71' 80  In  one  exten¬ 
sive  set  of  field  experiments,  the  correlator  was  operated  at  20  frames  per  second  while  recognizing  fly-by  aircraft 
images.811 

Recently,  BPOF  correlation  using  the  Texas  Instruments  Deformable  Mirror  Device  (DMD)  SLM  have 
been  reported.**1  This  electrically-addressed  device  has  128  x  128  element  resolution,  cycles  readily  at  TV  frame 
rates  and  higher,  and  is  capable  of  continuous  phase  modulation,  which  was  not  used  in  these  initial  experi¬ 
ments. 

Also  recently,  the  magneto-optic  SLM  devices  have  been  shown  to  provide  easily-accessed  ternary  (1, 0,  -1) 
modulation  levels,  and  simple  TPAF  correlations  showing  enhanced  signal-to-noise  were  demonstrated.7-1 


5.3  Real-Time  Joint  Transform  Correlation  Experiment 

Real-time  joint  transform  correlation  also  requires  real-time  SLM  devices,  with  slightly  different  perform¬ 
ance  requirements.  The  input  and  reference  patterns  are  introduced  side-by-side  in  the  input  SLM  and  a  buffer 
zone  between  them  usually  must  be  allowed  so  the  desired  correlation  component  and  other  terms  are  separated 
at  the  output  plane.  Thus,  the  space-bandwidth  product  (that  is,  resolution)  of  the  input  SLM  must  be  2  to  3 
times  that  of  the  images  being  processed  if  a  single  SLM  is  used.  Alternatively,  two  adjacent  SLMs  may  be  used, 
if  practicable.  At  the  Fourier  plane,  an  interference  pattern  must  be  square-law  detected,  modulated  onto  a 
coherent  beam,  and  optically  Fourier-transformed  to  produce  the  correlation  pattern  (and  other  components). 
This  requires  an  optically-addressed  SLM,  or  an  imaging  photodetector  whose  output  signal  drives  an  electrical¬ 
ly  aduiessed  SLM.  The  resolution  of  this  SLM  must  be  2  to  3  times  that  of  the  processed  images,  but  there  is  no 
requirement  for  complex  (phase  and  amplitude)  modulation  since  the  signal  is  square-law  detected. 

As  mentioned,  the  use  of  four-wave  mixing  in  nonlinear  crystals  to  form  a  real-time  filter77  may  be  viewed 
as  a  form  of  joint  transform  correlation. 

Real-time  joint  transform  correlation  has  been  implemented  using  the  Hughes  Liquid  Crystal  Light  Valve 
SLM  as  the  Fourier  plane  device  and  a  magneto-optic  SLM  as  the  input  device.8^ 

Figure  12  diagrams  an  experiment  using  a  liquid-crystal  TV  (LCTV,  the  familiar  inexpensive  consumer 
appliance)  for  both  input  and  joint  transform  modulation,  performed  sequentially.81  The  adjacent  input- 
refcrence  pattern  was  first  introduced  and  the  resulting  joint  transform  pattern  detected  by  the  vidicon  and 
recorded  on  tape.  This  signal  was  replayed  into  the  same  liquid-crystal  TV  and  transformed  to  generate  the 
correlation  pattern.  Preliminary  results  indicated  successful  correlation.  The  ultimate  utility  of  the  liquid- 
crystal  TV  for  coherent  optical  processing  depends  largely  on  the  degree  to  which  phase  distortions  normally 
exhibited  can  be  reduced  in  improved  devices  or  ameliorated  by  other  techniques.8*1 


5.4  Binary  Joint  Transform  Correlator 

Reports  of  the  experimental  demonstration  of  binary  joint  transform  correlation  are  likely  in  the  near 
future  since  two  of  the  leading  SLM  types,  the  magneto-optic  SLMs  and  the  deformable  mirror  device,  appear  to 
be  good  candidates  for  implementing  binary  joint  transform  modulation. 


5.5  Real-Time  Fourier-Feature  Processor 

A  Fourier-fcature  processor  derives  its  output  based  on  features  consisting  of  spectral  components  of  the 
spatial  Fourier  transform  of  the  input  pattern,  and  thus,  does  not  strictly  fall  within  the  scope  defined  for  this 


section.  Nevertheless,  the  process  is  closely  related  to  correlation  and  a  notable  el  fort  has  been  underway  to 
develop  real-time  commercial  systems  for  machine  vision  inspection.^-*  Global  Holonetics  Corporation,  Fair 
field.  Iowa,  is  developing  such  systems  using  ring-wedge  detectors  to  read  out  radial  and  a/imuthal  Fourier 
features  which  are  then  processed  (non-optically)  using  appropriate  discrimination  techniques  to  derive  the 
desired  classification  between  acceptable  or  reject  items,  for  example,  a  misprinted  or  misaligned  label  on  a 
container. 

6.  CONCLUSION 


Hardware  and  analytical  advances  in  the  last  five  years  have  led  to  a  rapid  evolution  of  real-time  correlation 
architectures  having  high  potential  for  practical  application  to  military  and  commercial  pattern  recognition 
problems.  With  the  potential  to  perform  correlation  of  reasonably  high  resolution  (for  example,  512x512  pix¬ 
els)  images  at  rates  approaching  1,000  frames  per  second  (for  both  input  and  reference  patterns),  in  packages 
well  under  a  cubic  foot  in  volume,  there  is  little  doubt  that  this  technology  can  provide  performance-to-si/.e 
ratios  orders  of  magnitude  above  any  other  approach,  now  or  in  the  foreseeable  future. 

The  perception  of  this  potential  is  leading  to  considerable  research  on  both  the  fundamentals  and  applica¬ 
tion  of  correlation  to  problems  such  as  automatic  target  recognition,  signal  processing,  quality  control  and 
inspection,  and  robotic  vision. 

As  reviewed  here,  a  variety  of  real-time  correlation  architectures  and  filter  formulations  arc  being  advanced 
to  attack  the  various  applications,  and  this  is  appropriate  because  the  different  approaches  have  complementary 
strengths  so  that  no  single  technique  is  likely  to  prevail  in  all  applications.  A  generic  hybrid  (optical/electronic) 
configuration  in  which  optics  does  the  computing  and  electronics  performs  filter  storage,  control,  and  post¬ 
processing,  is  emerging  as  a  paradigm  for  the  optimum  configuration  of  real-time  correlation  systems  realizable 
in  the  near  future. 
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and  selected  acquisition  programs  In  support  o  f 
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